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[Document Name] Specification 

[Title of the Invention] Image Processing System 

[Claims for Patent] 

1. An image processing system comprising: 

synchronous type processing means for carrying out a first image process on 
image data that is the subject of processing; 

asynchronous type processing means for carrying out a second image process on 
image data of a predetermined region of said image data that is the subject of 
processing; and 

synthesize means for synthesizing an output of said synchronous type processing 
means and an output of said asynchronous type processing means to form one image 
data. 

2. The image processing system according to claim 1, wherein said synthesize 
means comprises 

a memory in which an output of said synchronous type processing means is 
stored, and 

replacement means for replacing a portion of an output of said synchronous type 
processing means stored in said memory with an output of said asynchronous type 
processing means. 

3 . The image processing system according to claim 2, fiarther comprising 
control means for controlling replacement timing by said replacement means by 
detecting status of an output of said synchronous type processing means stored in said 
memory. 

[Detailed Description of the Invention] 

[Technical Field to Which the Invention Belongs] 

The present invention relates to image processing systems, particularly to an 
image processing system including a synchronous type processing means and an 
asynchronous type processing means. 
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[Conventional Art] 

A conventional image processing apparatus is known that reads out image data 
stored in a memory to carry out iniage processing on the read put image data. The 
image processing apparatus can be formed of a synchronous type or an asynchronous 
type circuit. A hardware circuit such as the ASIC (application specific integrated 
circuit) is exemplary of the synchronous type circuit. A circuit that carries out a 
process with software formed of a MPU (microprocessor) and the like is exemplary of 
the asynchronous type circuit. 

[Problems to be Solved by the Invention] 

The conventional image processing apparatus imposes problems set forth in the 
following. 

In an image processing apparatus entirely formed of the synchronous type circuit, 
the processing speed is determined by the process of the heavy load. There was a 
problem that the process is slowed down even if it is not necessary to carry out a heavy 
load process. This is because a process corresponding to the time consumed for the 
heavy load process is required whatever image data is processed. For example, in a 
synchronous type circuit 900 shown in Fig. 1 1 including a line memory 901, a matrix 
processing unit 903 and a y correction unit 905, data delay occurs since image data of N 
lines must always be stored in line memory 901 . Also, since a complicated matrix 
operation process (for example, integration of floating point) is carried out, data 
(operational result) delay occurs. 

When the entirety is implemented by the synchronous type circuit, the 
synchronous circuit per se becomes complex and lacks flexibility. When the resolution 
or the like of the image data is altered, the entire structure of the apparatus must be 
modified, resulting in increase in cost. 

When the entirety of the image processing apparatus is implemented by the 
asynchronous type circuit, high speed processing is degraded. 

In view of the foregoing, an object of the present invention is to provide an 
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image processing apparatus that allows high speed processing, and that prevents 
increase in the cost when the resolution or the like is altered. 
[Means for Solving the Problems] 

According to an aspect of the present invention, an image processing system 
includes a synchronous type processing means for carrying out a first image process on 
image data that is the subject of processing, an asynchronous type processing means for 
carrying out a second image process on image data of a predetermined region out of the 
image data that is the subject of processing, and a synthesize means for synthesizing the 
output of the synchronous type processing means with the output of the asynchronous 
type processing means to form one image data. 

Preferably, the synthesize means includes a memory storing the output of the 
synchronous type processing means, and a replacement means for replacing a portion of 
the output of the synchronous type processing means stored in the memory with the 
output of the asynchronous typt processing means. 

Further preferably, the image processing system includes a controller controlling 
the replacement timing by the replacement means by detecting the state of the output of 
the synchronous type processing means stored in the memory. 

The asynchronous type processing means carries out a second image process 
only on image data of a predetermined region. Therefore, an image processing system 
that allows high speed processing is provided. When there is no predetermined region 
on which the second image process is to be carried out, only the output of the 
sjmchronous type processing means is to be used. Therefore, high speed processing is 
allowed. 

The asynchronous type processing means has flexibility. Therefore, it is easy to 
modify only the processing contents that require modification (for example N- 
dimensional process) when the resolution or the like of the image data is altered. Thus, 
increase in the cost can be suppressed even in the case where the resolution or the like is 
altered. 
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By carrying out the second image process through the asynchronous processing 
means during the process through the synchronous type processing means, the process 
can be further speeded. 
[Embodiments of the Invention] 

Fig. 1 is a diagram to describe an operation of an image processing apparatus 
according to an embodiment of the present invention. Referring to Fig, 1, an image 
process of light load is carried out on the entire region of document image data 100 by 
means of a synchronous type circuit. The result is written into a memory. An 
asynchronous circuit carries out a process such as matrix operation on regions (for 
example, text region) 100a and 100b in document image data 100 that require 
processing of heavy load. The processed result of the asynchronous type circuit is 
overwritten on the memory in which the processed result of the synchronous type circuit 
has been written. 

The synchronous type circuit carries out high speed image processing on the 
entire region of the document image data whereas the asynchronous type circuit carries 
out image processing exclusively on the region that requires heavy load processing such 
as a matrix operation. The process carried out by the S5mchronous type circuit includes 
the one-dimensional process such as y correction. The process carried out by the 
asynchronous circuit includes the two-dimensional process such as edge enhancement 
process on a text region and smoothing process on a halftone dot region. 

By virtue of the flexibility of the asynchronous type circuit, the processing 
contents can be modified easily when the resolution or the like of the image data is 
altered. Therefore, increase in cost can be suppressed. 

Fig. 2 is a diagram to describe in further detail the operation of the image 
processing apparatus. Referring to Fig. 2, the image of interest is stored in an input 
image memory having the capacity of WX pixels horizontally and WY pixels vertically. 
Data IDi of one pixel in the input image memory is read out in series and subjected to y 
correction with a predetermined fijnction to become y corrected data ODil. 
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Y corrected data ODil is recorded at a corresponding address in the output 
image memory, y correction is carried out on all the pixels. 

Then, a matrix operation is carried out on a pixel in a predetermined region in 
the input image memory. It is assumed that the address of the pixel that is the subject 
of matrix operation is (i, j), and data of that address is D (i, j). The matrix has the size 
of M laterally and N vertically. Data subjected to the matrix operation is represented 
by the following equation (1). 

TZk(iJ)-D(iJ) (1) 

where T, k (i, j) are constants, and i = 0 - (M -1), j = 0- (N-l). Then, y correction is 
carried out on this data to result in data 0Di2. 

First, y corrected data ODil is written into the output image memory, and then 
data ODi2 is written into a predetermined region. 

[First Embodiment] 

Fig. 3 is a block diagram showing a structure of an image processing apparatus 
according to a first embodiment of the present invention. Referring to Fig. 3, the 
image processing apparatus includes an input image memory 201, an input memory 
control unit 203, a y correction unit 205, an output memory control unit 207, a matrix 
operation unit 209, and an output image memory 211. For the process of input data, a 
data bus for input data is used. For the process of the output data, a data bus for 
output data is used. 

Image data input by means of a scanner and the like is stored in input image 
memory 201 . The storage capacity of input image memory 201 corresponds to WX x 
WY for a monochromatic image, corresponds to WX x WY x 3 for a color (RGB) 
image, and corresponds to WX x WY x 4 for a color (CMYK) image, when the size of 
the image data is (number of main scanning pixels, number of sub scanning pixels) = 
(WX, WY). For the sake of simplification, it is assumed that the storage capacity of 
input image memory 201 is WX x WY in the following. Although image data coding 
is not effected in the present embodiment, the memory capacity of input image memory 
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201 can be reduced by carrying out a coding process. 

Input memory control unit 203, y correction unit 205 and output memory 
control unit 207 are implemented by an ASIC circuit that allows high speed processing 
since the circuits are formed of relatively simple logics. Matrix operation unit 209 is 
formed of a MPU (microprocessor) that allows complicated operation to read out image 
data of M x N pixels and apply a complicated operation thereon. 

Under DMA (direct memory access) control, input memory control unit 203 
reads out image data stored in input image memory 201 sequentially from the start 
address to the end address (start address + WX x WY). Input memory control unit 
203 includes an internal register in which a status flag (imb flag) is held indicating 
whether access to input image memory 201 is busy or not. 

Y correction unit 205 converts and outputs the input data according to a table of 
output data (8 bits: 0 - 255) corresponding to the input data (8 bits: 0 - 255). 

Under DMA control, output memory control unit 207 sequentially writes the y- 
corrected data from the start address up to the end address (start address + WX x WY) 
into output image memory 211. Output memory control unit 207 includes an internal 
register in which a status flag (omb flag) is held indicating whether access to output 
image memory 21 1 is busy or not, 

Matrix operation unit 209 monitors the imb flag of input memory control unit 
203 to read out the data of M x N pixels including the target pixel by random access 
from input image memory 201 when the imb flag indicates a non-busy state. Then, the 
matrix operation such as edge enhancement is carried out on the text region and 
smoothing on the halftone region. Matrix operation unit 206 further carries out a 
process likewise the y correction formed of logic. Matrix operation unit 209 monitors 
the omb flag of output memory control unit 207 to write the resultant data of the matrix 
operation into the address of output image memory 211 corresponding to the target 
pixel when the omb flag indicates a non-busy state. Accordingly, y-corrected data from 
y correction unit 205 is replaced in output image memory 211. 
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The region on which matrix operation is to be carried out can be determined by 
matrix operation unit 209. Alternatively, a region determined in advance can be input 
through the data bus. 

The output image data subjected to y correction and matrix operation in output 
image memory 21 1 is provided to a printer engine and the like through the data bus. 

When image data is written into input image memory 201 in the present 
embodiment, input memory control unit 203, y correction unit 205 and output memory 
control unit 207 carry out a one-dimensional process of light load such as y correction 
on the image data in input image memory 201. Since the imb flag in input memory 
control unit 203 indicates a busy state during this process, matrix operation unit 209 
cannot read out data from input image memory 201. 

Following the y correction on all the image data in input image memory 201, 
matrix operation unit 209 extracts image data that is the subject of processing from a 
predetermined region of the image data stored in input image memory 201 . A two- 
dimensional process is appUed on the extracted image data. The processed result is 
overwritten on the predetermined region in output image memory 211. 

Fig. 4 is a flow chart for describing the process carried out by matrix operation 
unit 209. Referring to Fig. 4, initialization is carried out at step SI 01. At SI 03, the 
address of the target pixel that becomes the subject of processing in input image 
memory 201 is determined. At step SI 05, the access status (imb flag) of input memory 
control unit 203 is confirmed. Waiting is conducted when the imb flag indicates a busy 
state (S107). 

When the imb flag indicates a non-busy state, control proceeds to step SI 09 
where image data of the matrix of M x N centered about the target pixel is read out. 
At step Sill, the operation process of the M x N matrix is carried out. 

At step SI 13, y correction is carried out on the image data subjected to the 
operation process. 

At step SI 15, the access state (omb flag) of output memory control unit 207 is 
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confirmed. At step SI 17, waiting is conducted until the omb flag of output memory 
control unit 207 indicates a non-busy state. At step SI 19, the write address of output 
image memory 21 1 is confirmed by output memory control unit 207. This causes the 
writing of the data subjected to matrix operation to attain a standby state until y- 
corrected data is written into the address of the target pixel by output memory control 
unit 207. At step S121, the data subjected to matrix operation is overwritten in output 
image memory 211. 

At step SI 23, determination is made whether all matrix operation has been 
completed. The routine ends when YES. When NO at step S 123, control returns to 
stepS103. 

According to the above-described steps of SI 05 and 107, matrix operation unit 
209 reads out the image data fi-om input image memory 201 when input memory control 
unit 203 is not busy. Accordingly, the access of input image memory 201 by input 
memory control unit 203 is given priority. 

By the above-described steps of SI 15 and SI 17, the writing operation by matrix 
operation unit 209 is carried out when output memory control unit 207 is not busy. 
Accordingly, access to output image memory 21 1 by output memory control unit 207 is 
given priority. A memory storing the processed result of one line can be provided in 
output memory control unit 207. Output memory control unit 207 may access output 
image memory 211 only when the process of one line has been completed by output 
memory control unit 207 and the process of the next line has not yet been initiated. 

[Second Embodiment] 

The image processing apparatus according to the second embodiment of the 
present invention has a structure identical to that of the first embodiment. Therefore, 
description thereof will not be repeated. The image processing apparatus of the second 
embodiment executes the flow chart represented in Fig. 5 instead of the process 
represented in Fig. 4. The image processing apparatus of the present embodiment is 
characterized in that a halftone region or an edge region is detected and image 
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processing carried out on the detected region by scanning the image data stored in input 
image memory 201. 

Referring to Fig. 5, initialization is carried out at step S20L At step S203, 
determination is made whether input image memory 201 can be accessed or not by 
referring to the imb flag. Waiting is conducted until access is allowed. 

At step S205, a matrix of 5 x 5 pixels centered about the target pixel is extracted 
from the image data stored in input image memory 201. At step S207, determination is 
made whether the target pixel is an isolated point or not. When YES at S207, control 
proceeds to step S209 where a larger matrix of 9 x 41 pixels is extracted, and then 
determination is made whether that matrix corresponds to a halftone image or not 
(S211). 

When YES at S21 1, control proceeds to step S213 to carry out a smoothing 
process using the data of the 5 x 5 pixels. Then, control proceeds to step S215 to 
carry out a y correction. At step S217, the omb flag is confirmed to determine whether 
access to output image memory 21 1 is allowed or not. When access to output image 
memory 211 is allowed, control proceeds to step S219 to write the data into output 
image memory. At step S221, determination is made whether the process has been 
completed for all the data. When YES at step S221, the routine ends. 

When NO at step S221, control returns to step S203. Also, when NO at step 
S221, control proceeds to step S221. 

When determination is made that the target pixel is not an isolated point at step 
S207, control proceeds to step S223 to determine whether the subject is an edge image. 
When YES, control proceeds to step S225 to carry out edge enhancement. Then, 
control proceeds to step S215. 

When NO at step S223, control proceeds to step S221. 

The processing time by the matrix operation unit according to the present 
embodiment is set forth in the following. 

(processing time of target pixel when corresponding to halftone dot) = (matrix 
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read out time of 5 x 5 pixels) + (time for determining an isolated point) + (matrix read 
out time of 9 x 41 pixels) + (time for determining halftone dot) + (time for smoothing 
process of 5x5 pixels) + (time for y correction) 

(processing time of target pixel when corresponding to edge) = (matrix read out 
time of 5 x 5 pixels) + (time for determining an isolated point) + (time for edge 
enhancement process of 5 x 5 pixels) + (time for y correction) 

(processing time of target pixel when not corresponding to edge or halftone dot, 
and is not an isolated point) = (matrix read out time of 5 x 5 pixels) + (time for 
determining an isolated point) 

(processing time of target pixel when not corresponding to edge or halftone dot, 
and is an isolated point) = (matrix read out time of 5 x 5 pixels) + (time for determining 
an isolated point) + (matrix read out time of 9 x 41 pixels) + (time for determining a 
halftone dot) 

In view of the foregoing, the relationship of the processing time is established as 
follows: 

(processing time of target pixel when not corresponding to edge or halftone dot, 
and is not an isolated point) < (processing time of target pixel when corresponding to 
edge) < (processing time of target pixel when not corresponding to edge or halftone dot, 
and is an isolated point) < (processing time of target pixel corresponding to halftone 
dot). 

[Third Embodiment] 

Fig. 6 is a block diagram showing a structure of an image processing apparatus 
according to a third embodiment of the present invention. The image processing 
apparatus of the third embodiment corresponds to the image processing apparatus of the 
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first embodiment shown in Fig. 3 with a region determination memory 25 1 connected to 
the data bus. Determination of a region of the image data stored in input image 
memory 201 is recorded in advance in region determination memory 251. In the 
present embodiment, region determination memory 251 distinguishes whether one pixel 
of the image data stored in input image memory 201 corresponds to an edge region, a 
halftone region, or a region other than the edge or halftone region. 

Fig. 7 is a flow chart of the process carried out by matrix operation unit 209 of 
the third embodiment. Referring to the Fig. 7, at step S301, initialization is carried but. 
At step S3 03, the address of the target pixel is extracted. 

At step S305, determination is made whether the target pixel corresponds to an 
edge region or a halftone region by region determination memory 25 1 . When the 
target pixel corresponds to an edge or halftone region, control proceeds to step S3 07 to 
determine whether access to input image memory 201 is allowed or not through the imb 
flag. Waiting is conducted until access is allowed. At step S3 09, the data of the 
matrix of 5 x 5 pixels around the target pixel is read out. A smoothing process is 
carried out if the target pixel corresponds to a halftone region (S3 1 1). An edge 
enhancement process is carried out if the target pixel corresponds to an edge region 
(S321). Then, y correction is carried out on the data of the target pixel (S3 13). Next, 
determination is made whether output image memory 211 can be accessed or not 
through the omb flag. Waiting is conducted until access is allowed (S3 15). 

At step S3 17, image data is written into output image memory 211, At step 
S3 19, determination is made whether the process of all the data has been completed or 
not. When YES at step S3 19, the main routine ends. 

When NO at step S3 19, control returns to step S303. When NO at S305, 
control proceeds to step 319. 

The processing time by the matrix operation unit in the third embodiment is set 
forth in the following. 

(processing time of target of interest when corresponding to a halftone dot) = 
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(time for reading out determination result of target pixel) + (read out time of 5 x 5 
pixels) + (time for smoothing process of 5 x 5 pixels) + (time for y correction) 

(processing time of target pixel when corresponding to edge) = (time for reading 
out determination result of target pixel) + (read out time of 5 x 5 pixels) + (processing 
time for edge enhancement of 5 x 5 pixels) + (time for y correction) 

(processing time of target pixel when not corresponding to edge or halftone dot) 
= (time for reading out determination result of target pixel) 

In view of the foregoing, the relationship of the processing time is established as 
follows: 

(processing time of target pixel when not corresponding to edge or halftone dot) 
< (processing time of target pixel when corresponding to halftone dot) < (processing 
time of target pixel when corresponding to edge) 

[Fourth Embodiment] 

Fig. 8 is a block diagram showing a structure of an image processing apparatus 
according to a fourth embodiment of the present invention. The image processing 
apparatus of the present embodiment differs from the image processing apparatus of Fig. 
3 in that input image memory 301 and output image memory 311 are set as dual port 
memories. Accordingly, the process by the synchronous type circuit and the process 
by the asynchronous type circuit can be carried out in parallel to further increase the 
image processing speed. 

Fig. 9 is a flow chart of the process carried out by a matrix operation unit 309 
according to the image processing apparatus of the fourth embodiment. The flow chart 
of Fig. 9 differs from the flow chart of Fig. 4 in that the step of identifying the access 
status of the input image memory and the output image memory through the imb flag 



-.12- 



JP{10-317997} 



and the omb flag (steps S105, S107, SI 15 and SI 17 of Fig. 4) is deleted. This is 
because matrix operation unit 309 can access the memory any time without taking 
account of the access to the memory in the fourth embodiment. 

It is to be noted that the output result of matrix operation unit 309 must be 
overwritten after the output result of the synchronous type circuit is written into output 
image memory 311. Therefore, control is provided so that the writing status into 
output image memory 31 1 by output memory control unit 307 is identified at step S419 
of Fig. 9 to allow the operation result of matrix operation unit 309 to be written into 
output image memory 3 1 1 after the output result from the synchronous tj^e circuit is 
written. 

[Effects of the image processing system of the above-described embodiments] 
Fig. 10 is a diagram to describe the advantage of the image processing apparatus 
of the present invention. It is assumed that the relative ratio of the image processing 
time is y correction (one-dimensional process): text correction (5x5 matrix operation) 
= 1 : 25 and one page of image data is formed of 100 pixels. When the entire process 
is carried out by a synchronous type circuit, the processing time of one page is 100 
(pixels) X 25. When only the y correction is carried out by the sjmchronous type circuit 
as in the above embodiments, the processing time is 50 (pixels) x 25 and 25 (pixels) x 
25 in the case where the text is distributed for 50% and 25% in one page, respectively. 

In other words, the processing time becomes longer when the all the process is 
carried out by a synchronous type circuit since text correction (5x5 matrix operation) 
is carried out on all the pixels irrespective of the text distribution. In contrast, only the 
required portion is subjected to text correction by virtue of the usage of both a 
synchronous type circuit and an asynchronous type circuit as in the above embodiments. 
Therefore, the processing speed is improved. 

' In the above embodiments, the processes are executed within a unitary apparatus. 
However, the processes can be carried out by a plurality of apparatuses. For example, 
the synchronous processing unit and the asynchronous processing unit can be 
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implemented by separate apparatuses. 

A program for executing the above process (SlOl - S123) stored in a storage 
medium can be loaded into the matrix operation unit and executed therewith. 

Although the present invention has been described and illustrated in detail, it is 
clearly understood that the same is by way of illustration and example only and is not to 
be taken by way of limitation, the spirit and scope of the present invention being limited 
only by the terms of the appended claims. 
[Brief Description of the Drawings] 

Fig. 1 is a diagram to describe an operation of an image processing apparatus 
according to the present invention. 

Fig. 2 is a diagram to describe in further detail the process of Fig. 1. 

Fig. 3 is a block diagram of an image processing apparatus according to a first 
embodiment of the present invention. 

Fig. 4 is a flow chart of the process of a matrix operation unit 209 of Fig. 3. 

Fig. 5 is a flow chart of the process carried out by the matrix operation unit of an 
image processing apparatus according to a second embodiment of the present invention. 

Fig, 6 is a block diagram of an image processing apparatus according to a third 
embodiment of the present invention. 

Fig. 7 is a flow chart of a process of a matrix operation unit 209 of Fig. 6. 

Fig. 8 is a block diagram showing the structure of an image processing apparatus 
according to a fourth embodiment of the present invention. 

Fig. 9 is a flow chart of a process of a matrix operation unit 309 of Fig. 8. 

Fig. 10 is a diagram to describe the advantage of the image processing apparatus 
of the present invention. 

Fig. 1 1 is a block diagram showing an example of an image processing apparatus 
formed of only a synchronous circuit. 

[Description of the Reference Characters] 
201,301 input image memory 
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203, 303 input image control unit 
205, 305 y correction unit 
209, 309 matrix operation unit 
211,311 output image memory 
25 1 region determination memory 
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[Document Name] Abstract 
[Abstract] 

[Subject] To improve the processing speed in an apparatus that carries out a plurality 
of image processing on image data. 

[Solving Means] Image processing that allows high speed processing such as y 
correction is carried out by a synchronous type circuit on all the region in document 
image data 100. Image processing is carried out by an asynchronous type circuit such 
as an MPU on a region that requires matrix operation such as the text region 100a, 100b 
in document image data 100. By synthesizing the processed results, one image data is 
obtained. 
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